Method for selecting state of a reconfigurable antenna in a communication system via machine learning

ABSTRACT

A method for selecting the state of a reconfigurable antenna installed at either the receiver or transmitter of a communication system is provided. The proposed method uses online learning algorithm based on the theory of multi-armed bandit to perform antenna state selection. The selection technique utilizes the Post-Processing Signal-to-Noise Ratio (PPSNR) as a reward metric and maximizes the long-term average reward over time. The performance of the learning based selection technique is empirically evaluated using wireless channel data. The data is collected in an indoor environment using a 2×2 MIMO OFDM system employing highly directional metamaterial Reconfigurable Leaky Wave Antennas. the learning based selection technique shows performance improvements in terms of average PPSNR and regret over conventional heuristic policies.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority under 35 U.S.C. §119(e) to U.S.Provisional Patent Application No. 61/532,131 filed Sep. 8, 2011. Thecontent of that patent application is hereby incorporated by referencein its entirety.

STATEMENT OF FEDERALLY SPONSORED RESEARCH

This invention was made with government support under Grant No. 0916480awarded by the National Science Foundation. The government has certainrights in the invention.

TECHNICAL FIELD

The present invention relates generally to the field of multi-elementantenna systems. Specifically, the present invention relates to methodsfor efficiently using multi-element reconfigurable antennas in MIMOsystems.

BACKGROUND

In recent years, studies have shown that reconfigurable antennas canoffer additional performance gains in Multiple Input Multiple Output(MIMO) systems by increasing the channel capacity, diversity order andeven have been shown to perform well in the low SNR regimes. Thesereconfigurable antennas are capable of generating multiple uncorrelatedchannel realizations by changing their electrical and radiationproperties and are gradually making their way into commercial wirelesssystems. The key to effectively utilizing the reconfigurability offeredby these antennas is to select a state that provides improvement inreceived SNR, throughput or channel capacity (referred to as “optimalstate” herein) among all the states for a given wireless environment.

Reconfigurable antennas can be employed either at the transmitter or thereceiver, or at both ends of the RF chain. This flexibility can create alarge search space in order to find an optimal state for communication.The key bottleneck to exploit the full potential of reconfigurableantennas is the requirement of additional training to obtain the channelstate information corresponding to each beam pattern and/or thecombination thereof at the receiver and transmitter. Moreover, theeffect of node mobility to a different location, changes in physicalantenna orientation, and the dynamic nature of the wireless channel canrender previously found “optimal” states suboptimal over time. Thismakes it important for a wireless system to employ a learning algorithmto find the new optimal states and to maintain the highest possible SNR.

In order to be effective, an online learning algorithm for antenna stateselection (also referred to herein interchangeably as “selectiontechnique”) must overcome certain challenges, including:

1) Optimal antenna state for each wireless link (between a singletransmitter and a receiver location) is unknown a priori. Moreover, eachwireless link may have a different optimal state. A selection techniqueshould be able to learn and find the optimal state for a given link.

2) For a given wireless link, there might be several states which arenear optimal over time, based on channel conditions and multi-pathpropagation. A selection technique should provide a policy to balancebetween exploiting a known successful state and exploring otheravailable states to account for dynamic behavior of the channel.

3) For the purpose of real-time implementation in a practical wirelesssystem, a selection technique must employ simple metrics that can beextracted from the channel without large overhead or requiring extensivefeedback data.

4) The selection technique should require reduced training or reducedchannel state information to keep the overhead low in a practicalwireless system.

Previous work related to state selection is based on estimating channelresponse of each antenna state which required changing the standard OFDMframe format. However, as the number of states increases, the schemebecomes impractical. See, e.g., A. Grau, H. Jafarkhani, and F. DeFlavis, “A reconfigurable multiple-input multiple-output communicationsystem,” IEEE Transactions on Wireless Communications, vol. 7, no. 5,pp. 1719-1733, 2008. Selection techniques using second order channelstatistics and average SNR information have also been proposed by D.Piazza, M. D'Amico, and K. Dandekar in “Performance improvement of awideband MIMO system by using two-port RLWA,” Antennas and WirelessPropagation Letters, IEEE, vol. 8, pp. 830-834, 2009. Further H. Eslami,C. Sukumar, D. Rodrigo, S. Mopidevi, A. Eltawil, L. Jofre, and B.Cetiner, proposed training schemes with reduced overhead and comparedthese to exhaustive search techniques in “Reduced overhead training formulti reconfigurable antennas with beam-tilting capability,” IEEETransactions on Wireless Communications, vol. 9, pp. 3810-3821, 2010.Though some of these techniques were successful in showing the benefitsof multi-state selection and motivated the need for a selectionalgorithm, none solved the challenges mentioned above and were not trulyadaptive in operation and required additional parameter tuning toperform optimally. Previous work in learning for cognitive radios hasprimarily been focused on link adaptation. See, e.g., R. Daniels, C.Caramanis, and R. Heath, “Adaptation in convolutionally coded MIMO-OFDMwireless systems through supervised learning and SNR ordering,” IEEETransaction on Vehicular Technology, vol. 59, no. 1, pp. 114-126, 2010,and S. Yun and C. Caramanis, “Reinforcement learning for link adaptationin MIMO-OFDM wireless systems,” in GLOBECOM 2010, 2010 IEEE GlobalTelecommunications Conference, Dec. 2010, pp. 1-5 and channel allocationfor dynamic spectrum access in Y. Gai, B. Krishnamachari, and R. Jain,“Learning multi-user channel allocations in cognitive radio networks: acombinatorial multi-armed bandit formulation,” in 2010 IEEE Symposium onNew Frontiers in Dynamic Spectrum, IEEE, 2010, pp. 1-9.

It is desired to develop learning algorithms for antenna state selectionto address the above challenges to improve the performance of wirelesssystems and to investigate the feasibility of implementing suchalgorithms in a practical wireless system. The present inventionaddresses these needs in the art.

SUMMARY

The present invention addresses the above-mentioned challenges in theart by formulating the antenna state selection as a multi-armed banditproblem. The multi-armed bandit problem described by T. L. Lai and H.Robbins, “Asymptotically efficient adaptive allocation rules,” Advancesin Applied Mathematics, vol. 6, no. 1, pp. 4-22, 1985; V. Anantharam, P.Varaiya, and J. Walrand, “Asymptotically efficient allocation rules forthe multi-armed bandit problem with multiple plays-part I: I.I.Drewards,” IEEE Transactions on Automatic Control, vol. 32, no. 11, pp.968-976, 1987; and P. Auer, N. Cesa-Bianchi, and P. Fischer,“Finite-time analysis of the multi-armed bandit problem,” Machinelearning, vol. 47, no. 2, pp. 235-256, 2002, is a fundamentalmathematical framework for learning unknown variables. In its classicform, there are N independent arms with a single player playing arm i(i=1, . . . N). Each play of a single arm yields random rewards whichare i.i.d with a distribution of unknown mean. The goal is to design apolicy to play one arm at each time sequentially to maximize the totalexpected reward in the long run. T. L. Lai and H. Robbins in“Asymptotically efficient adaptive allocation rules,” Advances inApplied Mathematics, vol. 6, no. 1, pp. 4-22, 1985, studied thenon-Bayesian formulation and provided a performance measure of an armselection policy referred to as regret or cost of learning. Regret isdefined as the difference in the expected reward gained by alwaysselecting the optimal choice and the reward obtained by a given policy.Since the best arm cannot always be identified in most cases using afinite number of prior observations, the player will always have to keeplearning and the regret will grow over time. Then, the regret of apolicy after n selections is given by:

${\mu^{*}n} - {\mu_{i}{\sum\limits_{i = 1}^{N}\; {E\left\lbrack {T_{i}(n)} \right\rbrack}}}$

where:

μ*=max_(1≦i≦N)μ_(i)

μ* is the average reward for the optimal arm, μ_(i) is the averagereward for arm i, n is number of total trials, E[] is the expectationoperator and T_(i) is the number of times arm i has been sampled. It hasbeen shown by Lai and Robbins that the minimum rate at which regretgrows is of logarithmic order under certain regularity conditions. Thealgorithm of the invention processes the received data to select theantenna state that minimizes the regret over time.

In accordance with an exemplary embodiment, the method of selecting anantenna state for a multi-element reconfigurable transmitter and/orreceiver antenna in accordance with the invention includes the steps ofproviding a learning algorithm that optimizes unknown signal to noiseratios of possible signal paths between at least one reconfigurabletransmitter antenna and at least one reconfigurable receiver antennaover time over different antenna array states and setting the antennastate for the transmitter and/or a receiver antenna based at least inpart on the antenna states determined by the learning algorithm to leadto an optimized signal to noise ratio between the at least onereconfigurable transmitter antenna and the at least one reconfigurablereceiver antenna over time. In an illustrated embodiment, the learningalgorithm formulates selection of an antenna stat as a multi-armedbandit problem for learning the unknown signal to noise ratios of thepossible signal paths so as to maximize signal to noise ratio betweenthe at least one reconfigurable transmitter antenna and the at least onereconfigurable receiver antenna over time. In this exemplary embodiment,the learning algorithm implements an arm selection policy referred to asregret defined as the difference in an expected reward gained by alwaysselecting a path with an optimal signal to noise ratio and a path thatleads to an optimized signal to noise ratio over time. The learningalgorithm processes the received data to select the antenna arrayconfiguration that minimizes the regret over time. Thus, thetransmission data is first collected and then later processed throughthe learning algorithm to benchmark performance. When the data isreplayed on the computer implementing the learning algorithm, thelearning algorithm is used to continuously make decisions until all ofthe data is processed. When this selection technique is deployed on alive wireless device and live transmission data is transmitted, thelearning algorithm is active at every transmission and selects the stateof the antenna system per transmission or per data packet. Whether thelive data transmission is emulated or actual live data is sent throughthe wireless device, the techniques of the invention will always beactive and will make sequential decisions.

The proposed method is adaptive in nature and can adapt to changes inwireless channel conditions, wireless node mobility and antennaorientation. The proposed method also is less computationally intensiveand has low feedback requirements for practical implementation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a composite right/left-handed two port reconfigurable leakywave antenna composed of 25 cascaded metamaterial unit cells.

FIG. 2 shows the measured radiation patterns for selected states andtheir corresponding bias voltages at port 1 (Gain˜−3 dB).

FIG. 3 shows the average reward for each algorithm for three multi-armedbandit policies (UCB1, UCB1-Tuned, UCB1-Normal, ε-GREEDY) and heuristicpolicies (ESPT, Random) verifying the empirical performance of theselection technique of the invention applied to the designated links.

FIG. 4 shows the node positions in an experimental arrangement usingfour distributed WARP nodes.

FIG. 5 illustrates normalized regret (Regret(n)/n) versus packet numberfor all links.

FIG. 6 illustrates empirical CDF of post-processing SNR averages acrossall links.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

A detailed description of illustrative embodiments of the presentinvention will now follow with reference to FIGS. 1-6. Although thisdescription provides a detailed example of possible implementations ofthe present invention, it should be noted that these details areintended to be exemplary and in no way delimit the scope of theinvention.

Learning Algorithm

The methods described herein are influenced by the work done by P. Auer,N. Cesa-Bianchi, and P. Fischer as reported in “Finite-time analysis ofthe multi-armed bandit problem,” Machine learning, vol. 47, no. 2, pp.235-256, 2002, where arms have non-negative rewards that are i.i.d overtime with an arbitrary unparameterized distribution. The setup isconsidered where there is a single transmitter and M wireless receivernodes and both the transmitter and the receivers employ thereconfigurable antennas. The transmitter has a fixed antenna state andthe receivers can select from N available antenna states. This reducesthe problem to selecting an antenna state only at the receiver end whereeach receiver can select state i independently. The decision is made atevery packet reception n to select the state to be used for the nextreception. If a receiver node selects a state i and it is assumed thatthe transmission is successful, a random reward is achieved which can bedenoted as R_(i)(n). Without loss of generality, R_(i)(n) is normalizedas R_(i)(n)∈[0, 1]. When a receiver selects a state i, the value ofR_(i)(n) is only observed by that receiver and the decision is made onlybased on locally observed history.

The selection technique described herein is based on the deterministicpolicy UCB1 given by Auer, Cesa-Bianchi, and Fischer. To implement thispolicy, the average of all the reward values observed for state i up tothe current packet n denoted as R_(i)(n) and the number of times state ihas been played, n_(i)(n) are stored. The UCB1 policy is shown below asAlgorithm I.

Algorithm 1 UCB1 Policy (P. Auer, N. Cesa-Bianchi, and P. Fischer,“Finite-time analysis of the multi-armed bandit problem,” Machinelearning, vol. 47, no. 2, pp. 235-256, 2002) // Initialization n_(t), R_(t) ← 0 Play each arm at least once and update n_(i), R_(i)accordingly. // Main Loop while 1 do  Play arm i that maximizes  $\begin{matrix}{{\overset{\_}{R}}_{i} + \sqrt{\frac{2{\ln (n)}}{n_{i}}}} & (1)\end{matrix}$  Update n_(i), R_(i) for arm i end while

The ε-GREEDY policy is also implemented, which is a randomized policy,and the UCB1-Tuned policy of Auer, Cesa-Bianchi, and Fischer isimplemented, which has been shown to work better for practical purposes.In the ε-GREEDY policy, the arm with current highest average is selectedwith probability 1−ε and a random arm is selected with probability ε.UCB1-Tuned is a fine tuned version of UCB1 policy which accounts for thevariance measured independently across arms. In this policy, the upperconfidence bound of UCB1 policy is replaced by:

$\begin{matrix}\sqrt{\frac{\ln (n)}{n_{i}}\min \left\{ {\frac{1}{4},{V_{i}\left( n_{i} \right)}} \right\}} & (2)\end{matrix}$

where V_(i) is defined as:

$\begin{matrix}{{V_{i}(s)} \equiv {\left( {\frac{1}{s}{\sum\; R_{i,s}^{2}}} \right) - R_{i,s}^{2} + \sqrt{\frac{2\; {\ln (i)}}{s}}}} & (3)\end{matrix}$

when arm i has been played s times during the first t plays. Anothervariant of UCB1 policy known as UCB1-Normal is also implemented where itis assumed that the rewards are drawn from a normal distribution. TheUCB1-Normal policy proposed in Auer, Cesa-Bianchi, and Fischer is givenas Algorithm 2 below.

Algorithm 2 UCB1-Normal Policy (P. Auer, N. Cesa-Bianchi, and P.Fischer, “Finite-time analysis of the multi-armed bandit problem,”Machine learning, vol 47, no. 2, pp. 235-256, 2002) // Initializationn_(t), R _(t) ← 0 // Main Loop while 1 do  Play the machine which hasbeen played less than 8logn times  Otherwise, play arm i that maximizes  $\begin{matrix}{{\overset{\_}{R}}_{i} + \sqrt{16\frac{q_{i} - {n_{i}{\overset{\_}{R}}_{i}^{2}}}{n_{i} - 1}\frac{\ln \left( {n - 1} \right)}{n_{i}}}} & (4)\end{matrix}$  Update n_(i), R_(i) for arm i end whilewhere q_(i) is the sum of squared rewards for arm i.

Exhaustive Search With Periodic Training (ESPT)

We compare the proposed multi-armed bandit algorithms with periodictraining scheme which requires exhaustive search to acquire channelstate information corresponding to each antenna state at. In theperiodic training scheme of an exemplary embodiment, each receiver goesthrough a training phase where each antenna state is activated insequence by a processor implementing the ESPT scheme and the reward fromeach transmission is stored in an associated memory. The amount oftraining is defined by the training period T. Once, the training phaseis over, the receiver selects the state with maximum average during thetraining phase and continues receiving on that state. The training isrepeated every F packets, which is defined as the frequency of thetraining This can be viewed as the process of consecutive explorationand exploitation, except that the duration of the exploration andexploitation is fixed and exploration occurs across all the states.

Reconfigurable Leaky Wave Antennas

The Reconfigurable Leaky Wave Antenna (RLWA) is a two port antenna arraydesigned to electronically steer two highly directional independentbeams over a wide angular range. Initially proposed by the authors in D.Piazza, D. Michele, and K. Dandekar, “Two port reconfigurable CRLH leakywave antenna with improved impedance matching and beam tuning,” in3^(rd) European Conference on Antennas and Propagation, 2009, EuCAP2009, IEEE, 2009, pp. 2046-2049, the embodiment shown in FIG. 1 is acomposite right/left-handed leaky wave antenna composed of 25 cascadedmetamaterial unit cells. See also, D. Piazza, M. D'Amico, and K.Dandekar, “Performance improvement of a wideband MIMO system by usingtwo-port RLWA,” Antennas and Wireless Propagation Letters, IEEE, vol. 8,pp. 830-834, 2009. Moreover, the application of various combinations ofbias voltages “S” and “SH” controls the beam direction allowing forsymmetrical steering of the two radiation beams at the two ports over a140° range.

In order to characterize the effect of beam direction on the efficacy ofa wireless system with RLWAs deployed at both ends of a link, a subsetof states was selected to allow the beam to steer over a range of 140°in the elevation plane. FIG. 2 shows the measured radiation patterns forthe selected states for port 1 and their corresponding bias voltages(Gain˜−3 dB).

Experimental Setup and Results

In their experiments, the inventors used the Wireless Open AccessResearch Platform (WARP), an FPGA-based software defined radio testbed,and WARPLab, the software development environment used to control WARPnodes from MATLAB Rice University WARP project. http://warp.rice.edu.Four WARP nodes were distributed throughout the fifth floor of theDrexel University Bossone Research Center as shown in FIG. 4. By usingWARPLab, each of the nodes were centrally controlled to allow for thesynchronization of the transmission and reception process and to providecontrol over the antenna state selected at each of the nodes. Althoughthe nodes were controlled centrally for data collection purposes, thelearning algorithm was decentralized. Specifically, no informationduring the learning process was shared with the transmitter. Thoseskilled in the art will appreciate that the WARPLab software developmentenvironment may be replaced in exemplary practical embodiments by acentral processor (not shown) or a processor connected to each antennaat each transmitter/receiver (not shown) programmed to control thesystem nodes and to control the antenna state selected at each of thenodes in accordance with one or more of the machine learning algorithmsdescribed above. Also, a memory (not shown) associated with the centralprocessor or with each processor associated with an antenna may be usedto store the received transmission and reception data for processing bythe selected machine learning algorithm.

The performance of the RLWA was evaluated in a 2×2 MIMO system withspatial multiplexing as the transmission technique. For baselinemeasurements, each designated WARP node transmitter broadcasted packetsmodulated using BPSK. For each packet transmission, the receiver nodesstored channel estimates in memory and measured the post-processingsignal-to-noise ratio (PPSNR) by evaluating the error vector magnitude(EVM) of the received symbol constellations. EVM is defined as theinverse of the squared symbol estimation error. Furthermore, the antennastates for each node were switched after each packet until all 5possible antenna states between the transmitter and receivers weretested. This process was repeated until 200 realizations were achievedfor all state combinations and for each node acting as a transmitter.The beam directions in FIG. 4 correspond to the optimal state selectedmost often at each of the receivers when node 4 was transmitting. Thealgorithm described above is an online algorithm but the collectedchannel realizations were used corresponding to each state and thealgorithm was evaluated in post-processing. This is important in orderto benchmark the performance of different policies under the samechannel conditions and to make sure that channel conditions do not biasthe performance results.

The results for three multi-armed bandit policies (UCB1, UCB1-Tuned,ε-GREEDY) are presented in FIG. 3 verifying the empirical performance ofthe selection technique. Each sub-figure represents the average rewardachieved by all three policies for a given wireless link over 200packets. The upper bound is defined as the reward obtained by a geniewhich always selected the optimal state with perfect channel knowledgeof all antenna states. For most of the links in FIG. 3, it was foundthat both UCB1-Normal and UCB1-Tuned outperformed the other policies.UCB1-Tuned has been found to work better for practical purposes since itis not sensitive to the variance of the states. Also, ε-GREEDY did notperform well because ε-GREEDY explores uniformly over all states and canselect sub-optimal states more often, thereby reducing the averagereward. It is evident from the figure that among three instances ofε-GREEDY policy, the instance with highest ε performed the worst.However, were mobile users considered in this experiment, it is possiblethat ε-GREEDY policy will adapt better to substantial variations inchannel condition. As, mentioned in above, one of the major drawbacks ofthe periodic training with exhaustive search is that the optimaltraining period (T) and training frequency (F) is not known a priori foreach link and it is challenging to vary those parameters in the case ofnode mobility and high channel variability. It is observed that in thecase of static receivers, by varying the T and F, performance of theexhaustive scheme is significantly affected. As the training period isreduced and training frequency is increased, sub-optimal states are usedmore often which reduces the average reward in the long run. It can alsobe seen that the exhaustive scheme with higher training period andhigher frequency converges faster, but has overall reduced performancein the beginning These results show that the ability of MAB policies towork as hands-off, semi-blind techniques without requiring parametertuning.

Regret Analysis

It is desired to find the optimal policy which can minimize the regretover time and provide logarithmic rate of growth of regret over time. InFIG. 5, the normalized regret is shown for all multi-armed banditpolicies and also the exhaustive and random scheme. P. Auer, N.Cesa-Bianchi, and P. Fischer proved that the regret for UCB1,UCB1-Tuned, UCB1-Normal grows logarithmically in time. The ε-GREEDYpolicy yields regret that is linear in time. Since, there are randompolicies, the regret is averaged over 200 runs, each run with 200sequential trials for the random policies. It can be seen that theregret of all multi-armed bandit policies have very low regret ascompared to the ESPT and Random selection policies. UCB1-Tuned has theleast regret among the multi-armed bandit policies. FIG. 6 further showsthe empirical CDF of Post-Processing SNR averaged across all links forboth multi-armed bandit and heuristic policies.

The above techniques were applied to methods of selecting an antennaarray state for a multi-element reconfigurable transmitter and/orreceiver antenna by providing a learning algorithm such as thatdescribed above that optimizes unknown signal to noise ratios ofpossible signal paths between at least one reconfigurable transmitterantenna and at least one reconfigurable receiver antenna over time overdifferent antenna array states of the at least one reconfigurabletransmitter antenna and the at least one reconfigurable receiverantenna. The antenna array states for the transmitter and/or a receiverantenna is then set based at least in part on the antenna array statesdetermined by the learning algorithm to lead to optimized signal tonoise ratios between the at least one reconfigurable transmitter antennaand the at least one reconfigurable receiver antenna over time. Forexample, the learning algorithm may formulate selection of an antennaarray state as a multi-armed bandit problem for learning the unknownsignal to noise ratios of the possible signal paths so as to maximizesignal to noise ratio between the at least one reconfigurabletransmitter antenna and the at least one reconfigurable receiver antennaover time. The learning algorithm may further implement an arm selectionpolicy to minimize regret (defined as the difference in an expectedreward gained by always selecting a path with an optimal signal to noiseratio and a path that leads to an optimized signal to noise ratio overtime). The learning algorithm processes the received data to select theantenna array state that minimizes the regret.

The methods of the invention thus incorporate a learning algorithm forantenna state selection that it is practical for use in wireless systemsemploying reconfigurable antennas. It has been shown empirically thatthe multi-armed bandit problem is a useful online learning framework forantenna state selection in a practical wireless system. For a network offour nodes employing reconfigurable antennas equipped with five states,the learning algorithm improves the received PPSNR and thereby improvesthe achievable throughput of the system.

While the invention has been described with reference to specificembodiments, the description is illustrative of the invention and is notto be construed as limiting the invention. Various modifications andapplications may occur to those skilled in the art without departingfrom the spirit and scope of the invention as defined by the appendedclaims. For example, the methods described herein may be used in atraining mode for a wireless transmission system or during anoperational mode where live data is transmitted to select antenna statesand configurations in real-time or near real-time.

Therefore, it must be understood that the illustrated embodiment hasbeen set forth only for the purposes of example and that it should notbe taken as limiting the invention as defined by the following claims.For example, notwithstanding the fact that the elements of a claim areset forth below in a certain combination, it must be expresslyunderstood that the invention includes other combinations of fewer,more, or different elements, which are disclosed above even when notinitially claimed in such combinations. A teaching that two elements arecombined in a claimed combination is further to be understood as alsoallowing for a claimed combination in which the two elements are notcombined with each other, but may be used alone or combined in othercombinations. The excision of any disclosed element of the invention isexplicitly contemplated as within the scope of the invention.

The words used in this specification to describe the invention and itsvarious embodiments are to be understood not only in the sense of theircommonly defined meanings, but to include by special definition in thisspecification structure, material or acts beyond the scope of thecommonly defined meanings. Thus, if an element can be understood in thecontext of this specification as including more than one meaning, thenits use in a claim must be understood as being generic to all possiblemeanings supported by the specification and by the word itself.

The definitions of the words or elements of the following claims are,therefore, defined in this specification to include not only thecombination of elements which are literally set forth, but allequivalent structure, material or acts for performing substantially thesame function in substantially the same way to obtain substantially thesame result. In this sense, it is therefore contemplated that anequivalent substitution of two or more elements may be made for any oneof the elements in the claims below or that a single element may besubstituted for two or more elements in a claim. Although elements maybe described above as acting in certain combinations and even initiallyclaimed as such, it is to be expressly understood that one or moreelements from a claimed combination can in some cases be excised fromthe combination and that the claimed combination may be directed to asubcombination or variation of a subcombination.

Insubstantial changes from the claimed subject matter as viewed by aperson with ordinary skill in the art, now known or later devised, areexpressly contemplated as being equivalently within the scope of theclaims. Therefore, obvious substitutions now or later known to one withordinary skill in the art are defined to be within the scope of thedefined elements.

What is claimed:
 1. A method of selecting an antenna array state for a multi-element reconfigurable transmitter and/or receiver antenna, comprising the steps of: a processor executing a learning algorithm that optimizes unknown signal to noise ratios of possible signal paths between at least one reconfigurable transmitter antenna and at least one reconfigurable receiver antenna over time over different antenna array states of said at least one reconfigurable transmitter antenna and said at least one reconfigurable receiver antenna; and the processor setting the antenna array configuration for the transmitter and/or a receiver antenna based at least in part on the antenna array states determined by said learning algorithm to lead to optimized signal to noise ratios between said at least one reconfigurable transmitter antenna and said at least one reconfigurable receiver antenna over time.
 2. The method of claim 1, wherein executing the learning algorithm includes formulating selection of an antenna array state as a multi-armed bandit problem for learning the unknown signal to noise ratios of said possible signal paths so as to maximize signal to noise ratio between said at least one reconfigurable transmitter antenna and said at least one reconfigurable receiver antenna over time.
 3. The method of claim 2, wherein executing the learning algorithm includes implementing an arm selection policy and providing a performance measure, referred to as regret which is defined as the difference in an expected reward gained by always selecting a path with an optimal signal to noise ratio and a path that leads to an optimized signal to noise ratio over time, said learning algorithm processing received data to select the antenna array states that minimizes the regret over time.
 4. The method of claim 3, wherein regret after n selections is given by: ${\mu^{*}n} - {\mu_{i}{\sum\limits_{i = 1}^{N}\; {E\left\lbrack {T_{i}(n)} \right\rbrack}}}$ where, μ*=max_(1≦i≦N)μ_(i) μ* is the average reward for the optimal arm, μ_(i) is the average reward for arm i, n is number of total trials, E[] is the expectation operator and T_(i) is the number of times arm i has been sampled. 